Texture Overlay onto Deformable Surface Using Geometric Transformation

We propose a system that overlays textures onto the arbitrary deformable surface of an object in real time using HMD. In this system, we select a book as the object and documents as the textures. Such overlaying of textures is achieved by rendering warped textures onto a surface which is projected onto an HMD image using some geometric transformations. We assume that the surface of an object consists of curved sub-surfaces which can be generated by a 2D geometric transformation, so that we can warp textures using the matrix of 2D geometric transformation. In order to deform textures fitly, we select the fittest transformation of 2D geometric transformations which we choose in this system. Since the system computes the transformation matrix in each frame, the texture can be adjusted to the surface in real time even though an observer with HMD moves, turns a page, or deforms a page. Therefore, using this system, the observer can read digitized documents as if he reads real books.